1. Field of the Invention
The present invention relates to image processing apparatus, method and system in which a virtual environment for virtual reality is generated, and the generated virtual environment is shared by plural users.
2. Related Background Art
Conventionally, there has been provided a method to express a three-dimensional object and space and provide an observation image at an arbitrary position and direction.
In this method (1), the three-dimensional object and space are expressed by using shape model data such as polygon data, curved surface data or the like, texture data representing a surface attribute or pattern of the image, light source data and the like, and a view state (or appearance) of the space in the arbitrary position and direction is sketched on a display screen, whereby the three-dimensional space and object are reproduced.
In case of constructing a three-dimensional virtual environment by this method, there is a method to express elements (coordinate conversion data, shape data, surface attribute data, illumination and the like) composing such the virtual environment by means of a tree structure. This is because the space, the ground, buildings, rooms, furniture, illuminators, ornaments and the like composing the three-dimensional space are considered to be essentially in a hierarchical nested relation. For example, since an ornament on a desk is moved according to an arrangement of the desk, the ornament is in a relation depending on the arrangement of the desk. Therefore, it is often more convenient for the ornament to be relatively arranged based on a coordinate system in which the desk exists, thereby applying the data structure in which the arrangements of the elements are in the hierarchically dependent relation. As a method to do so, the virtual environment is expressed by a n-tree structure.
For example, FIG. 42 shows an example of one simple virtual environment. In this case, if it pays attention to a room, a desk and a sofa in the space, the room is described on a coordinate system C.sub.2 converted from a world coordinate system C.sub.0 through a coordinate conversion T.sub.2, and the desk and the sofa in this room are respectively described on coordinate systems C.sub.3 and C.sub.4 respectively converted from the coordinate system C.sub.2 through coordinate conversions T.sub.3 and T.sub.4. Further, a pot on the desk is described on a coordinate system C.sub.5 converted from the coordinate system C.sub.3 through a coordinate conversion T.sub.5. FIG. 43 shows a tree structure schematically expressing such relation.
Further, there is a method (2) in which the light space data is generated based on an image group obtained by real-photographing the three-dimensional object and space, wherein the visible image in arbitrary position and direction is formed based on the generated light space data, and the formed image is displayed to reproduce the three-dimensional object and space. In this method, a shape of an object is not obtained, but the object is reproduced as a set (or gather) of light.
Furthermore, there is provided a method to perform cooperative working by plural users.
In this method (3), a data memory device storing data common to the plural users and a server device for managing processing on the data memory device are provided, and each user communicates with the server device to refer to and change the data stored in the data memory device or each user directly communicates with other users with respect to the change of the data in the data memory device, thereby maintaining the data of each user in the same state.
Furthermore, there is provided a method (4) that, by utilizing the method (3) to perform the cooperative working and the method (1) to express the virtual environment, the state of the virtual environment consisting of geometrical shape data is maintained to be identical, whereby the virtual environment is shared by the plural users.
However, in the above-described conventional methods (1) to (4), there are the following problems.
In the method (1), it is difficult to generate or reproduce the shape data for an object having a shape which is highly complicated. Further, it is also difficult to capture, from a real object, the shape data of this object of having a complicated shape by using a three-dimensional measurement device. Especially, it is more difficult to reproduce the real object having the already-existent complicated shape or surface pattern or a complicated reflection characteristic (i.e., absorption transmission characteristic). In addition, generally, it is easy to express an artificial object but difficult to express a natural object. However, in this method, there is an advantage that the artificial and simple three-dimensional space such as the room or a row of houses (or street) which is mainly composed of different planes can be expressed by a small data amount. Further, the method to express the virtual environment by using the tree structure is an excellent method.
In the method (2), when the light space data and is generated from a photographed image of a certain amount, then an observation image at an arbitrary viewpoint position (although including restricted condition) can be formed. Since the shape of the object is not restored and the object is described based on the photographed image of the object itself, this method is an excellent method which can express the object in a highly realistic manner. However, since the large data amount is necessary and the photographing becomes difficult when the object has a wide range, this method is suitable to describe the three-dimensional object rather than the wide space. That is, this method is not suitable to express the three-dimensional space such as the room or the row of the houses.
Further, the above conventional methods (1) and (2) have the different features respectively. However, in the conventional method (4), the method or technique which is combined with the method (3) to realize the common virtual space is only the method (1) of which the data amount is small. That is, the method (2) could not be used in the method (4) because of its large data amount. For this reason, the common virtual environment realized by the method (4) comes to include the feature in the method (1), whereby the common virtual environment becomes the virtual environment which can not include the object such as the natural object having a shape which is highly complicated.